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SpaceGraph is a -new, cqi^uter-driven display technology capjabie of 
sfibw^^ true three-dimensional. The \ 

inst^ctor/o (IpSX of a modern f light simulator iias) 

severe display .problems from the complexity of. the simulation 

k syst:em and' i€s* actions;. This report details our findings on -how the new 
display re of 'SpaceGraph can be used in the IOS." 

Simply put, r despite differences , all IOSs share one common display 
need: the cWreht position and velocity of their simulated aircraft in 
-three-diTOnsiohaL space* Inherently co/itinjCious (as opposed to discrete) 
anytime varying r these data, are also-spatial. Yet they are shown to the 
instinactor/pperato on flat screens , a form of presentation dimensionally 
i^smatched to the data; This mismatch creates a greater workload on the 
instructor/operator through its inescapable awkwardness. 

We conclude that space-filling data should be shown with a space- 
filling ..display. Awkwardness would change to naturalness. The demand 
on the instructor/operator to integrate disparate flat presentations 
into a mental construct of performance in three-dimensional space and 
then to criticize -such performance would be replaced with an. intuitive 
presentation allowing, immediate criticism. 

Beyond the explicit goals of the contract , we have also observed 
that part of the training functions now carried on in the classroom and 
in the simulator could instead be done with a space-filling display 
which gives -an outside-in view of one or more aircraft. The apparent 
.benefit is decreased training coot and time* 

SpaceGraph embodies a new technology in computer-driven displays. 
Using a, novel form of time-varying, very-wide-field-of-view optics, the 
technology provides all of the virtues of interactive jcomputer graphics 
but with images that are space-filling* Images appear in a -display 
volume, not on a display surface. Further details on this visually 
dramatic technology are presented in the appendix. 

/ - ; ■ - J - 

Regarding SpaceGraph as a new display resource, it is natural to 
seek sites and applications, for it which are now not well served by 
flat displays* One such site is the instructor/operator station 
of a modern flight simulator* There the instructor and the operator of 
the simulator (the instructor sometimes may also be the operator) face 
a panoramic array of instrument dials and catho&ray tubes (CRTs), the 
totality pf^ which hopefully^ keejf>s them informed about trainee perform- 
ance. The purpose of this effort was to find ways at the IOS in which 
SpaceGraph could reduce the- i^trt^ctor/operator's ^workload* 
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The central thrust in this contract, as- Originally conceived, was to 
match "the information display needs of the IOS to the information dis- - :0 

play attributes „bf SpaceGraph* The needs were to be derived from the 
results of a previous. Air Force Human Resources contract. The attri- 
-;butei>%rere .nqt.-only .to, :be cataloged but were to be enhanced in several 
q ^way.s likelyHr :as then judged tb improve x the match of needs to 

m£ ' • - $. -; ■ . • - • . ; j 



attributes. As time passed, however, for a variety of reasons, the list 
of needs was not mde available in a useable form* We therefore took a 
less structured, more pragmatic approach: 1 We observed actions at two 
IOSs to: see first hand what problems SpaceGraph might ameliorate. We 
then erected sample images directed at those IOS problems. 

Despite .these .contractor-approved deviations from the originally pro- 
posed methodology, the effort has produced results more immediately 
applicable to current IOS problems — and to other facets of pilot train- 
ing than j*e expected. SpapeGraph, by showing an outside-in view in . 
trixe 3-D o.f one or more aircraft, ppens up a major new training vehicle. 
It bridges the gap between the 6jutside : in view of hand flying in the 
classroom and the inside-out view of simulator flying. 
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II*. lbs NEEDS And spacegraph capabilities 
IPS Needs 

We were able to ascertain some needs, by first-hand observation, at 
three IOSs: one at the SAAC* at Luke AFB and one each at the ASPT** and 
at the undergraduate pilot training facility at Williams AFB. After two 
days of observing, one simplistic observation emerged as paramount: 
Pfying is- intensively three-dimensional, yet none of the instructor's 
displays was other than flat, The instructors were mentally integrating 
the information from multiple displays, tneir mental constructs acting 
as surrogates for the display they did not have — i.e., an outside-in 
\'iew of the student's plane in 3-space. 

When displays are mismatched to the information to be conveyed, the 
inevitable results are a proliferation of displays and a higher than 
necessary workload. In this case, say the ASPT, one IOS display showed' 
the aircraft's position, while another showed its altitude versus one 
position coordinate. If no further information were necessary, there 
would already be two images showing four things, whereas the information 
content, considering the capabilities of the SpaceGraph display, justify 
only one image showing three things. 

There are fundamental questions left unanswered: Should the in- 
structor be able to criticize pilot performances using an error- 
measuring capability not available to the pilot? If the instructor 
uses displays not available to the pilot, can the instructor translate 
criticism into a form useful to^the pilot? Would the student' pilot 
benefit from seeing the instructor's displays in a replay debriefing? 
Our workscope led us to formulate such questions but did not begin to 
. ,!£0Re necessary to answer them in a formal way. Informally, 
however'i^it^appeafed ; tq us that the needs of the instructor and those 
of the student are not cleanly separable. v 

Considering the large number of parameters monitored and displayed 
or ,,4 ls Playable at the I QS, we expected at least several needs to be 
apparent to the instructors and/or operators. In conversations with at 
lease four IOS personnel, we found that not to be the case. Instructors 
and operators, presumably proficient at their IOS posts, had learned to 
cope and expressed satisfaction. Their jobs were obviously demanding 
but ; nqt, in their- perception, too demanding. They did not express dis- 
"J*^ 8 **^ 10 * W- 'a»Pect of IOS data display, believing, apparently, 
that; since everything; conceivable -was displayed or displayable, they had 
no grounds for complaint. 

Instructors' awareness of training needs in the classroom was a 
different story. Apparently hand flying is the primary display, and it 



"Simulator for Air-to-Air Combat 
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wis 4eeraed lacking in many ways. For example, it fails for formation 
fiyingV&r turns and rolls not compatible with arm joints, and, in 
general, it results in a realism gap, (Notably missing from the de- 
ficiencies was the outsige-in view it provides.) 

We also discussed training n^eds during debrief ing/ replays . At the 
SAAC st\tf mtT~see a TV-like replay- Its flat presentation of a non- 
flat subject was deemed undesirable but the outside- in view was not- 
An important suggestion was made there (at Luke AFB) that an off-line 
display (i.e. , one with ^stand-alone capability) of air-to-air combat 
could be used as a game in the lounge. Also suggested by the instruc- 
tors w(5 interviewed was a replay capability which could show a student's 
improvement ever time. 

SpaceGraph Capabilities 

Unless specifically to the contrary, the following discussion refers 
to easily achieved capabilities of the generic technology pioneered by 
the current implementation, not to capabilities of the current implemen- 
tation itself. 

The appendix discusses in detail the modes of SpaceGraph operation: 
A, B, and C. (These modes were formerly referred to, respectively, as 
line mode, surface mode, and volume mode, a terminology which was poten- 
tially confusing.) 'Briefly, mode A is suited to draw space-filling 
images made of dots, lines" and alpha-numerics. Mode B is best suited to 
draw space-filling images which can be thought of as landforms, i.e., 
vertical deformations of a horizontal plane. Mode C is best suited to 
draw space-filling images which can be thought of as cloudforms, i.e., 
3-D scalar fields (which have a real numeric value at each lattice 
point (x, y, z) in the volume). In general, mode A has high spatial 
resolution for sparse images while mode C has low spatial resolution for 
nonsparse images. Mode JJ has some features of A and some of C. Mode A 
is clearly the mode of choice for depicting airplanes, coordinate 
reference frames, alphanumeric flight parameters, runwa^ight^T^d^tr^i 
plots, and other sparse images which want high spatial resolution. ■ 

As a resulc of this contract, a new mode was implemented: overlay/ 
underlay (0/U) mode. It is intended for creating mode A-like images in 
the front-most plane of the display volume, called overlays, and in the 
rear-most plane of the display volume, called underlays* The purpose is 
to expand the display's capability by taking advantage of the two other- 
wise useless times during each of the mirror's vibrating cycles when the 
mirror's velocity is zero. 

As a second result of this contract, mode A was made operational 
during both halves of the mirror's sinusoidal cycle. This new capabil- 
ity immediately doubles the achievable complexity of images in mode A. 
' However-, exploration of its capability must await a new mirror, since 
the; present one fails, to achieve sufficiently perfect fores troke-back- 
stye ke sy m m etry , fin improved mirror is being developed under other 
funding* 
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As a third result of this contract, a new ineaps fc was designed and 
brought near to operational status whereby the viewer can easily direct 
a .pulsed laser beam into the image for the dual purposes of pointing and 
of selecting light buttons. 

Mixed mode operation is using the two halves of the mirror cycle in 
two different modes ..—An example of a mode A/modc B mix is an airplane 
in mode A fTying..over. terrain in mode B. To this pair could be added an 
O/U-mode image of an altitude scale,, or a wind shear diagram", or^alpha- 
numeric captions, instructions, or data. 

Although SpaceGraph might reasonably be described as a. true 3-D 
display, it is important to note that the three Ds referred to are only 
the spatial dimensions. Along with other CRT displays, SpaceGraph also 
has brightness and time variations as display resources. So it would 
not be unreasonable to describe SpaceGraph as a 5-D display and other 
conventional CRT displays as 4-D. No matter how described, however, 
SpaceGraph has one more dimension than conventional CRTs, and therein 
lies its unique capability. ^ 

The central goal of this contract is to find ways at the IOS for 
which SpaceGraph 1 s unique capability can be advantageously exploited. 
It is tempting to restrict the search to the depiction of space-filling 
arrangements of objects and scenery. But to be more systematic, Space- 
Graph's capability should be kept in mind: one more dimension. There- 
fore, applications now pursued on flat screens, e.g., plotting range 
versus fuel, can be extended, .e.g. , plotting range-versus fuel versus 
altitude. In general, any one or mppe of SpaceGraph 's five dimensions ' 
(x, y, z, brightness, time). can beoised to plot a spatial (e.g., alti- 
tude) or a non-spatial (e.g., lb. of fuel) parameter.- It is easily seen 
that the number of combinations of spatial and non-spatial parameters is 
very large, so new constraints can. be usefully introduced. One such 
constraint is that a spatial parameter must be plotted by x, y, or z. 
Another, on the grounds of simple pragmatism, is that the image should 
embody its. information content in an intuitive form. (It, is possible ta 
imagine plots which are very informative but only after a lengthy "accli 
mati?ation. ff It is more than possible to imagine such plots as suc- 
cessful in theory but ignored in practice.) 

As a final observation on SpaceGraph's capabilities in general, we 
must not overlook the obvious: SpaceGraph can show aircraft ^realis- 
tically flying in 3-space, while flat presentations cannot. Sinfce air- 
craft flying is the central subject, SpaceGraph 1 s one added dimension • 
could be the key to a whole new (additional) technology for flight 
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HI. IDEAS AND TRIALS • * • , 

Because the central task had a large element of brainstorming, we 
sought ideas from as many jLnforaed\people'as possible/ Here follows a, 
list of thesis ideas, in their raw form, /organize^ chronologitally by 
meeting number* , Contributors at these meetings were Bolt Beranek and 
Newmart Inc. staff members and consultants, Air Force personnel*, and 
Sanger- Link Simulator personnel. ' All contributors were qualified by 
virtue of being pilo.ts, instructor pxlpts, supervisors of pilot training, 
researchers in -pi lot training, or experts in human factqrs as they re- 
late to flying.- The line items are^ numbered for later referencing: 

° 

Ideas fot ways to use a SpacteGraph displdy at (and in conjunction with) 
the IPS - I 3 - ; ' ' ^ L 

— — — — • 4 * , 

Meeting #1 * T % 

e \ 

1. Monitor heading, altitude; and airspeed together t 

2. Show landing approaches * 

3. - Show interceptor tactics ^ 
4* Air-to-air refueling ... 

S* Formation flying in global view (outside-in) 
- 6. Remotely Piloted Vehicle (RPV) - 

7. ' Ground Controlled Approach 

8. Performance analysis 

9. /Instrument/night air work and navigation. 

10. / Predictive displays like flying through. Instrument Landing 

I System gates 
11/ Coordinated maneuvers 



Meeting #2 

-k 

12 \ Flightpath analysis (plane represented as a point) with 
reference to: ^ 

airspace restrictions 
i speed restrictions 

landing pattern 

lj3 v Parameter and parameter sets which would profit from a true • 
3-D representation: - • 

airspeed 
altitude 

turn and slip > 0 
attitude 
heading 

\ Instrument Landing System (ILS) approach 

indications 
Very-high-frequency Omnidirectional Range 

indications 
climb and descent with rates of change 
14. ILS gates with airplane and velocity vector 
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Meeting #3 

15/' Summary information not readily shown in a flat picture 
t 16. Approach path taken 

17. Aerobatics; velocities and accelerations 

18. Landing maneuvers within the air space restrictions 

19. Air-to-air combat 

Cannon orientations 
20. , History of flight track improvement 

Meeting #4 

21. Air-to-air combat 

22. One plane flying relative to terrain 

23. Landings 

Give instructor a variable viewpoint (outside-in) of landing 
Slow motion, "stop frame" capability 

Meeting #5 

24. Approach control geometry 

Desired location relative to actual location 
Predicted path 

25. Parametric display of flight maneuvers, e.g., 

elevator (as x), rudder (as y), aileron (as z), 
simultaneously plotted at -successive times 

Meeting #6 

26. Outside-in view instead of inside-out as a part- task trainer 

27. Effects of wind in outside-in view 

28. Gliding as a form of anfcrgy management 

Parameter displays showing relations of air speed, 
sink rate, glide ratio, time aloft, maximum range 

29. Predictive displays for carrier approach and landing 

30. Dogfight game theory 
Si.. ^Terrain following 

Meeting #7 

32. Sets of displays as a training manual of maneuvers, correlated 
with control settings 

33. Aerial combat with more than two aircraft — outside-in view 

34. Energy maneuvering 

Meeting #8 ^ 

35 . Energy/ management 40* 

36. ILS 7 c^ne with aircraft (outside-in view) 

37. Air refueling rendezvous and docking considerations 
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38. Airborne uses for true 3-D display 

View of own and other aircraft showing cone of weapons 

influence 
RPV 

39. Terrain following 
Outside- in view of fixed plan over moving terrain 

Gliding— vectors with impact prediction 



40 

Meeting #9 



41. Enhancement to existing part-task trainers 

Compare performance to that required in basic tactical 
game skills 

42. Replays of performances in real aircraft from telemetered data 

Performance analysis showing time versus critical task- 
dependent flight parameters 

Meeting #10 

43. History of student performance 

44. Debriefing display for instructor and student, showing 
o-'' actual versus desired performance 

Meeting #11 

45. Replacement for current displays which show 3-D flightpaths 

with multiple 2-D views 

46. Aid to teaching judgment by showing parametric data more 

pictorially, e.g., formation flying 

Meeting #12 

47. Lower-cost training device than T-37 or ASPT 

48. Judgment training aid for Landing Signal Officers 

49. m Ordnance delivery 

50. Energy management 

51. Substitute for classroom hand-flying (outside-in view) 

52. Many-plane interactions 

Combat tactics 
Formation flying 

53. Replay/debriefing using data telemetered from actual aircraft 
Meeting #13 

~ 54. A/r-to-air combat (outside-in view) 
As game for practicing 
As aid for debriefing 

As pictorial presentation of usually digital data 
air speed, angle of attack, accelerations, altitude, 
control surface usage 

55. Missile, cannon fire trajectories 

14 



56. - Zoom- in outside-in viewing showing, with high pictorial 

resolution, the relative orientation and separation of 
air combatants when close* 

j 

Meeting #14 

57. Aid to instructor's jiidgmejit in showing better the student's 

control settings 

58. Integrated displays showing aircraft plus control settings 

and energy information 

Meeting #15 

59. Replacement for hand flying in showing effects of wind 

60. Predictive display for carrier landing 

61. Training in tactical game theory 

62. Training in terrain following 

Meeting #16 

63. Flying-manual augmentation by showing maneuver in real 3-space 

64. Pattern analysis of errors in coordinated maneuvers 

65. Airborne substitute for inside-out view under low visibility 

conditions 

66. Energy management training aid 

67. Outside-in view to show student pilots where they are in a 

specific maneuver- 

Meeting #17 

68. Airborne radar imagery showing own plane and weapon cones 

69. Airborne use particularly ttractive for bombers 

70. Terrain following for helicopters 

71. Gliding 

Predictive displays 

Space shuttle landing trainer 

This list conveys several new and important concepts: 

1. Hand flying in the classroom gives an outside-in view while 
full-blown simulators give an inside-out view. There appears 
to be an enormous gap here, one in which a SpaceGraph display 
might be a remarkably attractive and cost-effective, outside-in 
part-task^ training aid. 

2. The IOS now is burdened with flat displays attempting to 
convey flightpath information. This information could be 
conveyed more simply and intuitively by a single true 3-D 
display. It is difficult, to imagine a more perfect match 
between the needs at the IOS and the capabilities of the new 
display technology. 
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3. Review and debriefing following simulator flight sessions 
might be helped by an outside-in view, i.e., by giving the 
student off-line the same critical aids used on-line by the 
instructor . 

Energy management was mentioned by three or four people as a topic 
^needing better visual, presentation. In order to better understand the 
subject, since no one we asked could give us a really clear exposition 
of it> we did a modest literature search. The result was that we now 
understand why it is both difficult to explain and important to under- 
stand. First, there seem to be two parts to energy management which are 
distinguished by whether or not the weight of fuel changes during the 
.time period of interest. Thus, one part couJd be called (but it isn't) 
"long-term energy management^ and the other "short-term energy manage- 
ment." Both subjects are treated theoretically in the literature in an 
academic style far removed from a flier 1 s world of moment-by moment 
decision making. It would appea\s£hat pilots in training are exposed 
to these twin subjects academically in class and practically while 
flying, in the hope that the connection will become evident as exper- 
ience accumulates* Our emerging view of this training problem is that 
to approach the extent that experienced pilots have learned the opera- 
tionally useful energy management ideas from flying, student pilots can 
learn with a.: outside-in view of a flyable plane. In short, intuition 
can be trained with a part-task special purpose trainer. 

Outside-in versus inside-out views emerged during these meetings as 
an importan* and fertile topic. There was much discussion over the ex- 
tent to which flyers think of themselves as if from outside and behind 
their own plane. For if this outside-in mental construct is indeed an 
important aid in 1 flying, the inside-out view of a simulator achieves 
realism at the' expense of good teaching technique. A flat display is 
best suited to an inside-out view of a semi-infinite airspace, while a 
SpaceGraph display is better suited to an outside-in view of a finite, 
piece of the world. So, it may be that SpaceGraph is very well matched 
to just that piece of pilot training not well served by conventional 
display technologies. 

Of the many other interesting ideas contributed at these meetings, 
at least two deserve comment. First, the possible use of a SpaceGraph 
display in the training of Landing Signal Officers (LSOs) is tantaliz- 
ing, since LSO training inherently requires zn outside-in view (as from 
a carrier deck), LSOs must undergo a lengthy training period including 
much on-the-job training, top LSO performance is vital to expeditious 
and safe boarding of Navy fliers, and LSO training aids are almost non- 
existent. 

Second, the idea for a- landing trainer for the Space Shuttle epito- 
mizes the landing, trainer problem: How can & pilot's reaction to a 
complex spatial problem be trained without endangering the pilot or the 
aircraft? An v answer is certainly suggested by observing the naturalness 
with which a true 3-D display Tenders spatially complex data sets. 
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In order to expedite the creation of trial images, two new computer 
programs were. created (they were beyond the proposed workscope). The 
first, called GDP (Geometric Description Processor), makes it possible 
to describe an object, like an airplane, with the primitive display 
elements of lines and dots and then to declare that the object is also a 
primitive. A new image can then be described as composed of lines, dots, 
and airplanes. The second new program, called PATHS (for paths in 
space), allows a smooth flightpath in space to be described by a series 
of connected straight line segments which the program then smooths with 
circular splines. The airspeed and turning radius of an aircraft can 
be specified along the path as thus defined, and the program will manage 
banking appropriately as well as display the aircraft at specified time 
increments. This multiple display of a single aircraft is in lieu of 
displaying a moving aircraft, which is not yet possible due to the early 
developmental state of the current system. 

Since we were directly exposed to the IOSs at the ASPT and at the 
SAAC, we concentrated on ways to use a SpaceGraph display at those sites. 
As already noted, each IOS attempts to display a three-dimensional air- 
craft position with one or two flat pictures. 

In the photographs (Figures 1 to 6) which' appear at the end of this 
report, it is crucial for understanding to realize that front/rear am- 
biguity, present in the photographs, is not present in the actual dis- 
played 3-D image, where front appears at the front and rear appears at 
the rear. Photographs cannot convey the content of a true 3-D image any 
more than can a presentation on a flat CRT. Here, however, we have no 
choice but to document our work photographically with the depth dimension 
squeezed out. The photographs are not of the CRT of the SpaceGraph 
display but rather of the same display volume seen by the viewer. This 
distinction is apparent in Figures 3 and 4, which are photographs taken 
of the identical image but with a change in camera position. Image 
quality is much better in the real 3-D image than in the photographs. 

Our first task was to create an image of an airplane, so that GDP, 
using it as pne of its primitive display elements, could scale it, orient 
it, and position it to suit the immediate needs. Figure 1 shows the air- 
plane model. It is sparse when shown at this magnification, but ad- 
equately dense when demagnified, using GDP and placed into a coordinate 
reference frame. 

The IOC at the SAAC has a perspective presentation of the two train- 
ees' planes. Figure 2 is a Xerox copy of a hard copy made from this IOS 
presentation. Figures 3 and 4 are two,vieti8 of one 3-D image which de- 
picts the same kind of pictorial information. 

At this point it is important to observe that the information in the 
SAAC from which Figure 2 was- created contains all of the relevant 3- 
space coordinates. To create Figure 2 therefore is a geometrical pro- 
cess of squeezing out one dimension to make it displayable on a flat 
surface. The information in the SpaceGraph display's computer from 




which the image photographically represented by Figure 3 (or 4) was cre- 
ated is the same set of relevant 3-sna.ce coordinates. In a few words 

inforSSn er ^' ,e "- t !? e SAA °l S an <* ^"Graph's, started with the same 
HII^ITa- ^ P01 ?^ u USt be ei PP h asized, since it implies that a 
SpaceGraph display could be attached to the SAAC rather simply. 

Figure 5 is a photographic representation of a 3-D image showing a 

Son eXeCU r ng the b ° mbing run at the ™ e =ame pLitionaT ijfor- 

Ta °k 1S .u h0Wn in tW ° Views at the I0S ° f ^e ASPT, one a rrJan view 
no? on,v r ^ 6leVati0n VieW ' " We believe that 'the single 3-D x'mage 
Somh ^-'T 1101 ^ 5 Simply ' bUt that ^ le nds itself to adding 

LT on ? ? ° neS i Predictive vectors, error-from-prescribed-paS infor- 

Sor^tJor°aTl 0 L??L h impr °r ement ' ° ther P°^ ti0nal and'attitudinal 
mrormation, all before resorting to alphanumerics. 

»^«! igUre f\ is a Photograph of an image we created for training or 
eSSIFf PH i ng format ion flying. Confusion in the photographs 
£ ^ ely absent / n the image, a fact which underscores (f now in- 
adequate flat surfaces are for these kinds of portrayals, and (blhow 
much information the 3-D image is lost when'one dimension is sqj^ed 

As a game in the lourige (suggestion #54) , it is exciting to imagine 

SSSL? K nti u g (in 2 " D) What each combatant sees (inVS) S! 

opponent could be the computer (much as one can play chess versus a 

S?K ) iiiSt 1 S2 ru ^ r, - or another student pi?ot The ,=t 

Scent ^^sri^^*™*™*™" 1 **' aerial tact i cs , °r radar 
inn?^ ^i ',, Mlsslles ^V^nnon fire could be shown. Ground targets 

n v 2 fen* ^ S Its^f^ c Tfr kS C0Uld , be Sh ° Wn - This kind of outside- 
m view lends itself to a wide range of possibilities. 

tice W b,i a tL^ nStrUC ! e ? n ° trial imag6S specifically for landing prac- 
s \ZuT+- t a f " 6aSt SeV6ral kinds Possible. In addition to 
simplistic outside-m images like that suggested by Figure 5 it might 
be Profitable to show details of the correct landing p^ern"- the 
glides ope, range markers, gates, etc. Inside-out ?iews are also worth 
Z ollTT?' alth o u |k they raise new questions about showing sceneS 
wJicTlLfbev^nr;, k f° S ? ibly attrac tive solution is to show scenery 
on thi hLk n? " baCk P lane - of the display volume as a 2-D picture 
™ hs ? ack P lane ~ 3ust as stage sets show 3-D props on stage and a 
flat backdrop of what lies beyond. 8 

showf rgoal^anowfhj^ 65 V"? f ° rSakeS realism and ^tead 

e^i! g c ' al j? wable .error bounds, and the current state. A simple 

£3 ! IL k g a PP roach: I^gine the goal to be a 3-D "bulSeye « 
!nTLft^L a v Ve and K bel ° W sh °w glides lope bounds; error bounds right 

bounds ! r n ? , 0Und K ; and err ° r b0unds fore a " d aft show velocity 
bounds. A 3-D cross-hair shows the current state of glideslope line- , 
up, and velocity. The error bounds naturally shrink Is touchdown is 
vS P W C t ed - K N ° tiCe , that the threG S P atial pensions of ?he display 
STne o?v b ? en ^ Sed i n tHiS eXam ' ple t0 show two dimensions of ?pace 
S s nfo e OClty ; Numerous goal directed tasks lend themselveS to 
this kind of presentation, but- we have not pursued them yet, believing 
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that images portraying greater pictorial realism will be easier for new 
users to accept* (Also, there is a question of applicability at the 
IOS.) Nevertheless, we believe the possibilities here are at least as 
great- as those in the photographs. \ 
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IV. PRIORITIES FOR FUTURE DEVELOPMENT 



We originally proposed tfc create a priority-ordered list of desirable 
future developments. Accordingly, here is the list as motivated by our 
perception of the requirements for pilot training in general and for the 
JOS in particular. 

1. Speed-up creation of display files: Adding motion to show airplanes 
flying requires much faster creation of display files than is cur- 
rently possible. We believe the speed-up required is about lOOOx, 
and it appears that the specificity of the application will make 
this possible,' even in the presence of the unfavorable (for this 
application) computer architecture surrounding the present imple- 

. mentation of jipaceGraph. 

2. Increase plotting speed: We believe this development to be impor- 
tant, since it will noticably improve the resolution of small air- 
plane images. This development will "be incorporated into the next 
generation of this display technology, 

3. Automatic phase synchronization between mirror and CRT: This re- 
finement over the current state-of-the-art is a prerequisite for 
an easy-to-use display device. It will be incorporated into the 
next generation of this display technology. 

4. Mirror refinement: This refinement inevitably will take place as 
succeeding mirrors are built. 

5. Increased display depth:- This improvement would facilitate several 
of the promising application ideas disclosed herein. However it is 
not necessary. Increased depth can be ,achiev$d by refinements in 
the mirror design itself or by auxiliary optics. Both approaches 
are promising, although the latter approach irs more straightforward. 

6. Increased mirror and/or CRT size: The need for large* mirrors or 
CRTs has not been established. It is an application-dependent 
question. The applications suggested herein can be adequately 

, served, at least at first, without major changes in geometry. Two 
possible exceptions: a wider CRT would help some applications, 
like those^ which portray migrating objects; and a smaller CRT/mirror 
combination might be advantageous for certain airborne applications. 

7. Daylight-viewable display: Because most of the applications sug-. 
gested herein can use a non-daylight-viewable display, this devel- 
opmental priority appears low. But there are some applications 
which must have a daylight-viewable capability, and for the moment, 
these applications cannot be seriously contemplated. Daylight view- 

* ability seems feasible, but it would require a substantial develop- 
mental effort with no guarantee of success. 



8. Phosphor optimization: This desirable task need not be undertaken 
as long as there are no serious problems with image brightness. 
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So far, there are not. 

9. Addition of a machine-erected cursor to the image: For almost all 
of the applications discussed herein, this refinement is not neces- 
' sary. However a cursor is indispensable for certain other applica- 
tions, sd that it will probably be developed anyway. 

10. Add color: Color 1 capability has been placed at the bottom of the 
priority list, since we are not convinced that color would mater- 
ially improve any of the application ideas discussed in this report. 
Color is a difficult enhancement for SpaceGraphT 



v; CONCLUSIONS 



The proposal for the work done under this contract was witten in 
April 1978, at which time the SpaceGraph display was less capable than now 
and application ideas in aviation had been limited to air traffic control. 
As direct results of this contract , werhave learned much about the IOS, 
the SpaceGraph display is technically more mature, and several application 
ideas for the IOS and for pilot training have crystallized. In particular 
we have identified specific IOS needs which could be directly met by the 
SpaceGraph^ display technology. 

Both at the SAAC and at the ASPT, the IOS uses awkward (in our 
opinion) presentations for aircraft position. These presentations 
appear to >e attempting exactly that which the SpaceGraph display does 
very well. 

• : ' ' 

* Beyond these clear-cutrways to use the new display resource at the 
IOS, we have begun to see a host of other possibly more important uses 
near the IOS. The one we regard with greatest immediate expectations, 
is the realistic outside-in view of one or more aircraft which can be. 
flown with realistic flight dynamics. The possibilities here for part- 
task pilot training seem remarkably broad, and considering the likely' 
operating cost of such a device, say $18/hj>*, remarkably inexpensive. 

Also very "promising is the exploitation of the third spatial dimen- 
sion to create a new kind of "flight 11 director for multi- variable' goal- • 
oriented tasks_, e.g., radar intercepts, air refueling, landing, ord- 
nance delivery, etc. This class of use is not near-term, however, 
since even the basic human factor work has not yet been done. 

Basic to any exploitation of the SpaceGraph display is having one 
and having that one with adequate performance. Currently one exists, 
and it is used by several different groups at BBN for very 'diverse 
purposes* Its performance is currently limited to showing static images. 
Although it could be reprogrammed to show simple moving images, like 
one or two aircraft, the next generation of this display technology will 
be much more suitable for this purpose. 



♦Assumptions: Capital cost of display plus host computer « $125,000. 

Total operating cost * 2.5% of capital cost per month. 
Usage rate 40/hours/week. / 



Figure 1. Photograph of a space-fllliDg 
image showing airplane model to be used 
as a primitive display element in subsequent 
images. Transparency of the airplane image 
ruHults in front-back confusion in the photo- 

ERXC ph but not ' n the actual 3 " D toage * 
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Figure 2. Hard copy of CRT display now 
used at the IOS of the SAAC at Vike AFB.' 
The locations of the airplanes in space are 
no clearer in the actual flat display than in 
this flat reproduction of it. 



Photograph of a space-filling '< 
rich presents in true 3-D. the 
rofiwb airplanes, these location: 
rent from the actual display but ! 
thlg j^J^)graph of it. 



.Figure 4. Photograph of the same/epacer 
filling image, used to make figure 3 . The 
" ^erence DeWeen figures 3' and 4 is that 
the camera was repositioned . 

ERIC 



l&yp'gare ft, $^<£o§^|Hl:, a space-flUlng 
|fe j^age which presents intxjae 3-1) the. " ' . f'j 



successive locations of the plane simulated 

It 9 . 



Figure 6. Photograph of a space-filling 
* ' . i$ag» which shows, with none of the. con- 
£•/_ ' P resent * n the 2~B photograph, how 
ir a particular aerobatic maneuver might be 
\ ■ 9 "orined. 



APPENDIX A 



THE SPACEGRAPH DISPLAY: PRINCIPLES OF OPERATION AND APPLICATION 

The SpaceGraph display provides space-filling images in a display 
volume rather than flat images on a display surface. Its use is logical 
and natural in 3-D applications now using flat displays, and its unique 
Visual properties mako possible altogether new applications. The 
purpose of this appendix is to explain how the SpaceGraph display works 
and how its capabilities can be exploited. 

Principle of Operation — Visual Perceptual 

Were it possible to oscillate a CRT along its axis, its phosphor 
screen would repeatedly sweep through a volume — the "display volume. 11 
Denoting screen coordinates by x and y and screen position in the volume 
by z(t), then an image element can be written at any (x, y, z) by 
writing it at (x, y) at the correct time. The image elements appear- 
ance will be satisfactory if 

(a) it is rewritten regularly every 33 msec or more often 
(>30 Hz refresh rate) 

(b) it is seen within a dark display volume 

(c) it is not smeared in z by phosphor persistence. 

Since it is not physically practical to oscillate a usefully large- 
screen CRT through a useftil depth at a 30 Hz rate, the visual equivalent 
is used — oscillating an optical image of the phosphor screen. 'Ms 
alternative is physically practical, since the CRT can be stationa y 
and the time-varying optics can take a very simple form. 

Principle of Operation — Optics 

As shown in Figure A-l, the simplest possibly suitable optical 
arrangement consists of an oscillating plane mirror. Because of the 
equality of image and object distance, the "leverage 11 of the plane mirror 
is 2, i.e., the image moves 2x as far as the mirror.* For a good view 
of the image, the mirror must be large compared to the CRT. "(Consider 
a 2- v inch-square bathroom. mirror: your image would be at the same place 
and have the same size as usual, but your view of the image would be 
severely restricted.) A plane mirror, therefore, must be large, must 
move through half of the desired image depth, and must oscillate at 
30 Hz. This- set of requirements does Yiot admit a simple solution. 
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Figure A-l. In principle, a true 3-D display could be achieved using an 
oscillating plane mirror* Such a mirror is shown here 
moving from its closer (to the CRT) to its further position* 
The image of the CRT undergoes a corresponding excursion, 
but with twice the amplitude of motion ♦ In practice, this 
scheme is essentially unrealizable* 



Compared to the case of the flat mirror, if the mirror is forced to 
deform, it is possible to reduce the amplitude of mirror motion without 
sacrificing the amplitude of image motion* The price is constancy of 
magnification (always unity for the flat mirror). Figure A-2 shows the 
idea, exaggerating the mirror deformation for clarity. 
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image will appear 
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Mirror deformation extremes 
* (exaggerated for clarity) 

In principle and in practice, a true 3-D display can be 
achieved by using a deforming mirror. The dimensions shown 
are used in the analysis below; Note that h is time-varying* 
and, consequently, so is q. Th* solid and dotted depictions 
of the mirror give rise to the solid and dotted image posi- 
tions respectively. 
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Here, the leverage is ^85, as can be easily derived. L<t distances 
measured to the left of the mirror be >0, and to the right <0. 



p * object distance (fixed) 
qCt) « image distance 

h(t) ■ amplitude of mirror deflection at center 
Assuming that the mirror is a spherical cap of radius r(t), or at 
least osculates with such a sphere, then it is known that 

p q(t ) yrtj 

Simple geometry shows that* 

r(t) ■ mn c? + h(t)?3 » 



and for the typical case wherein |h max | <0. Old, 



r(t) * BhTtT 
Therefore, 

« (t) " rShTtT i 

3^"P * 

For sinusoidal time dependence, where h(t) *-|h max | sin wt, 

1 . 



q(t) 



TBh 



max 1 
pP^ sin cot - - 



For simplicity, it is convenient to let A - ■ ^ ax ^ and B « i , so that 

<* (t) " A sin^t - B 
Typical values ar * e | n maxl " °' 2cm » d " 30cm, and p « 67cm, so 
A -.3.56 x 10"* and B - 1.49 x 10~ 2 . 

For the convex extreme, sin tut « -1 and <5 convex * -5*. 1cm. 
For the concave extreme, sin cot » +1 and °. concave * -87.9 cm. 



*Note: The minus sign arises becuase the sign convention of Figure A- 
requirds that h(t) and r(t) always have opposite signs. 
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Thus the extremes of image position are 33.8 cm apart, a seeming viola- 
tion of intuition considering that the mirror is moving only 0.4 cm 
(peak-to-peak) at its center. A potential disadvantaged that the 
image magnification, always given by |q/p|, is non-constant, varying 
from 1.3. (concave case) to 0.81 (convex case). However, a compensatory 
magnification on the CRT is; easily implemented. The net inescapable 
price one pays is a display volume whose shape is the frustum of a rec- 
tangular pyramid (see Figure A-2). 

Principle of Operation — Acoustical 

The key pitfall in the design of a vibrating- mirror display is the 
inadvertent production of acoustic noise levels ranging from unpleasant 
to intolerable. Perceptually, the energy spectrum tells all. The ear 
is notably insensitive to frequencies at or below about" 30 Hz. But 
40 Hz is already more perceptible. Therefore it is important tot keep - 
the mirror motion as purely sinusoidal, at or below 30 Hz, as possible. 

In the present design/ two fictors contribute to quietness. First, 
the mirror is designed as a mechanically resonant structure, with its 
fundamental resonant frequency (in the mode of interest — one circular 
node) at the desired frequency* of operation. This arrangement is energy 
efficient and has clean sinusoidal behavior. Second, since the circular 
plate mirror vibrates with one concentric circular node, the edge of the 
front surface recedes as the center advances, and vice versa. This 
behavior reduces the alraady low radiation efficiency, of the plate, 
whose diameter is only 3.5% of the wavelength jof 30 Hz sound in air. 
Also, the visual effect of extending the mirror beyond its supporting 
hinge is extremely favorable, since the viewer has more freedom of head 
movement . 

Principle of Operation — Mechanical 

A vibratory structure whose deformed shapes at resonance are used 
optically and whose acoustical output is potentially malignant is a 
delicate design problem. . For that reason, we resorted to a .numerical 
model of the structure so as to iterate toward an optimal structure 
without many expensive construction/trial cycles. 

Mechanical driving power ^for the plate is supplied by air pressure 
from an abutted woofer. Only a few watts are necessary, even for large 
plates (e.g.* 40 cm), since the acoustic power radiated from the plate 
(and from the. woofer cone's back surface) is very small, and the air 
coupling from-woofer" to resonant plate is efficient. Ocher drives are 
possible, since so little power is required. and since sinusoidal be- 
havior is the native behavior of a resonant structure. However, one 
must be very creative to match the reliability and low cost of a 
commercial woofer. 

Principle of Application 

The control of the x or y axis of a CRT is always "plot" or "sweep." 
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With M plot, ,f the x-position (say) is set to a specific value, a value to 
be_ maintained until further notice. With, "sweep, 11 the x-position is 
.caused to assume all legal values sequentially, usually in a linear 
motion. Since a raw CRT has two spatial axes, and each can operate in 
.either of these two modes, thsre are four possibilities. Eliminating 
plot x/sweep ~y, since it is simply a symmetrical variant of sweep x/plot 
y, there are three essentially distinct graphics modes and all are in 
common use: % 

plot x/plot y, a "vector" or x,, directed beam" display device; 
sweep x/plot y, a "time-base oscilloscope"; and 
sweep x/sweep y, a "raster 11 display device. 

If the swept spatial axis created by the moving mirror is called z (the 
brightness axis, sometimes called z, here will be called b), then> a 
direct extension -of the foregoing categories of possible graphics modes 
is as follows: 

Mode A: plot x/plot y/sweep z 
. Mode B: sweep x/plot y/sweep z 
Mode C: sweep x/sweep y/sweep z 

This three-way distinction, illustrated in Figure A-3 is pivotal in the 
following discussion. 
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Figure A-3. To display a long image element, Modes A, B, and C involve 
increasing amounts of waiting. 

The essential idea is that a volume element at (x; y, z) can be lit 
up to a specified brightness by one of three methods. The diagram makes 
clear that Mode C could entail a lot of waiting. Waiting is anathema t6 
a refresh-type display, since there is a fixed interval of about 33 msec 




in which to draw the image. Therefore, Mode C should be used only when 
there are image elements at most of the possible (x, y, z) locations. 

For the same reason, Mode B should be used only when there are 
image elements at most of the possible (x, z) locations (there is no 
waiting for y). 

It follows that Mode A is suited to sparse images, 

, Complementing this 'simple temporal argument is a simple spatial 
argument. A refresh- type display needs a memory out of which repeatedly 
flows the image information. -For excellent spatial definition of a line 
drawing M Mode C, the image lattice should be at least (1000, 1000, 500). 
With 4 bits for brightness, the corresponding memory size is 2 x 10 9 bits 
and the corresponding data rate is 6 x 10 10 bits /sec, putting this use 
of Mode C well into infeasibility. For excellent spatial definition of 
the same line drawing in Mode A, the image memory needs 28 bits per 
plotted value (12 for x, 12 for y,^4 for b) and, as will be shown, about 
15000 values. There results a memory size of 420,000 bits and a data 
rate of 12 x 10 6 bits per second, wholly feasible numbers and with a 
better visual result in the bargain! 

In summary, Modes A, B and C have complementary strengths and weak- 
nesses. Mode A can provide excellent spatial resolution for a sparse 
image, while Mode C overcomes the spars eness limitation at the expense 
of spatial resolution. Mode B offers some virtues of A (resolution in 
y) and some of C (non-sparseness in x and z). 

. Use of Mode A 

As z is swept, a sequence of (x, y) values is plotted on the CRT. 
The image memory only stores the sequence of (x, y) values, the "display 
file," since z is implicit in the order. 

For purposes of creating the display file from a simple logical des- 
cription of the desired image, the "raw picture," it is possible to im- s 
agine the raw picture as existing in some number N of depth zones. If 
N were small, each zone would have^ more, in it, and the spatial resolu- 
tion in z would suffer. The interesting question is how to size N. A 
simple argument, based solely on the above observation, leads inescap- 
ably to the conclution that N should be as large as possible. The 
largest possible value of N is such that each such depth zone contains 
the minimal possible image element — a dot. SpaceGraph's mode A works 
in that wayi plotting one -dot in each of many (thousands) depth zones. 

Creating lines from dots means that the number of dots is the figure 
of merit rather than the more traditional figures of merit based on the 
nunfcer and/or lengths* of long and ?hort vectors. Since 10- dots, by 
observation, makes 1 cm of vector (in the display volume, not on the 
CRT), a conversion from 15000 dots to length of vector gives, 1500 cm. 



By visual, trials, this length of vector has been found to be sufficient 
for most applications. 

Use of Mode B ^ 

Mode B essentially draws profiles and stacks thera in depth. Terrain, 
for example as defined by contour lines, is easily converted to profiles 
and shown in mode B. in general, this mode, is suited for any scalar 
function of two independent variables fe.g., altitude as a function of 
latitude and longitude). Using brightness in addition, one can show an 
additional scalar function (e.g., temperature and altitude as a function 
of latitude and longitude). . 

A different use of brightness is to hide hidden surfaces which other- 
wise would be visible. Hidden line elimination in the .usual 2-D sense 
does not work here, since a line may or may not be hidden depending on 
the viewers head movement. t We have observed that an excellent expedient 
is to gradually (in space) dim out the line as it approaches a perman- 
ently hidden- place ^ 

Use of Mode C 

Mode C naturally lends itself to the display of three-dimensional 
scalar fields — e.jj. , isotope concentration at each lattice point in 
a volume, or the density to X-rays for each volume element in a patient's 
head. The presentation provided by Mode C is (a) objective, (b) free of 
the artifacts of slicing, and (c) in a form admitting interaction with 
tho viewer. Here, "objective 11 implies that two viewers see the same 3-D 
image, as opposed to seeing the same set of 2-D images and then indepen- 
dently making mental constructs to 3-p. "Free of the artifacts of 
slicing 11 implies that a solid s?nere is displayed like a solid sphere, 
not like a set of /solid disks. "In a form admitting interaction" im- 
plies that the viewer can .reasonably ask to see and then g^t to see a 
revised presentation, for example, a spatial vignettie excluding non- 
essential .and possibly-obscuring foreground and background imagery. In- 
teractivity distinguishes SpaceGraph images from holographic images. 
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